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(54) NOTICING AREA EXTRACTING DEVICE AND AUTOMATIC COMPOSITION 
DECIDING DEVICE 
(57)Abstract: 

PROBLEM TO BE SOLVED: To extract a noticing area 
adjusted to hunnan subjectively and to automatically 
device well-balanced composition. 
SOLUTION: A noticing area extracting device and an 
automatic composition deciding device 10 include an 
image forming device 14, which generates the original 
image of a panoramic image from video photographed 
by a camera 12. A noticing area extracting device 20 
extracts a noticing area from the original Image given 
from the device 14. Namely, evaluation matched with 
human subjectivity is given in accordance with the 
physical feature of the original image and the noticing ^ 
area is extracted in accordance with the evaluated result. 
A composition cutting off device 22 cuts out the 
extracted noticing area and an adjacent image area from the original image by referring to 
data on paintings painted by painters and photographs taken by photographers stored in a 
memory 24. Namely, the data can be cut out by the same composition as the case with 
painted images or photographed images. 
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* NOTICES* 

Japan Patent Office is not responsible for any 
damages caused lay the use o£ this translation. 

1 .This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2.**** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] . 

[Industrial Application] This invention relates to the attention field extractor cut out firom a subject-copy 
image so that the composition of a picture that experts, such as a painter and a photographer, made the 
attention field which extracted and extracted the attention field may be suited and it may be especially 
settled, for example fi'om a subject-copy image about the automatic composition decision equipment 
which used an attention field extractor and it, and the automatic composition decision equipment using 
it. 

[0002] Here, the field where people observe an attention field in an image or an image is said. 
[0003] 

[Description of the Prior Art] As this kind of conventional technology, various methods are proposed 
about the technique to which a watcher extracts an attention field fi-om an image. (1) Milanese and Itti 
And Koch and others assimies the discontinuous part in an image to be an attention field, and generates 
the feature map (shade image) corresponding to two or more physical features acquired fi-om an image, 
next asks for the discontinuous part of each feature map, and is extracting as an attention field what 
imified them. 

[0004] (2) Milanese and others filtered the feature map using two or more Difference-of-ofiented- 
Gaussians filters with which magnitude was fixed in the discontinuous part of each feature map, and 
chose and searched for the filtering result fi-om which an output serves as max. Itti, Koch and others 
normalized each feature map with the square error with each average, and xmified all the feature maps by 
linear combination. And the integrated feature map was recursively filtered with the Difference of 
Gaussian filter, and the local peak of the filtering result finally obtained was extracted as an attention 
field. 

[0005] Thus, Milanese, Itti and others were extracting the attention field by processing of pixel level, 
such as filtering and relaxation. 

[0006] (3) Martin, Takeuchi and others evaluated the brightness information acquired fi-om an image 
based on the information theory of Shannon, and made the attention field the portion with the high 
amount of information acquired as a result. By this method, the field, i.e., the field which looks 
complicated, where distribution is large and the bright field of a brightness value are mainly extracted. 
[0007] Moreover, with the conventional camera, human being had determined the composition of a 
photographic subject and a photographic subject manually. 
[0008] 

[Problem(s) to be Solved by the Invention] However, since attention area size changed with images by 
the technique of of (1) and (2), it was difficult to extract an attention field exactly using the filter with 
which magnitude was fixed. Moreover, in the case of the picture which cannot not necessarily be said to 
be in agreement [ the field which looks compUcated, or a bright field ] with an attention field like the 
black picture of a flower vase placed, for example in firont of the wall of a compUcated pattem by the 
technique of (3), it was difficult to extract an attention field exactly. Furthermore, by such proposal. 
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there were few examples which performed collating with a watcher's (human being) subjectivity and an 
extract result, and it was a question whether the attention field which actually suits human being's 
subjectivity can be extracted. 

[0009] Moreover, with the conventional camera, when ordinary men without the sense over a 
photograph determine composition, the good photograph of balance cannot necessarily be taken. 
[0010] So, the main purpose of this invention is oflfering the attention field extractor which can extract 
exactly the attention field which suited human being's subjectivity. 

[001 1] Moreover, other purposes of this invention are offering the automatic composition decision 

equipment which can determine the good composition of balance automatically. 

[0012] 

[Means for Solving the Problem] The 1st invention is an attention field extractor which extracts an 
attention field from a subject-copy image, and is an attention field extractor equipped with an evaluation 
means to evaluate the degree of [ ** ] based on the physical feature (whenever for it to be conspicuous), 
and an extract means to extract an attention field according to an evaluation result of an evaluation 
means. 

[0013] The 2nd invention is automatic composition decision equipment which used an attention field 
extractor of a publication for the 1st invention, and is automatic composition decision equipment 
equipped with a maintenance means to hold data about a criteria image which has criteria composition, 
and a cutoff means which cuts out an image of an attention field firom a subject-copy image with 
reference to criteria composition. 
[0014] 

[Fimction] An evaluation means evaluates the degree of [ ** ] by the attention field extractor of the 1st 
invention according to the physical feature of a subject-copy image. Here, the degree of [ *♦ ] means the 
parameter suitable for human being's subjectivity. An extract means extracts a most conspicuous field 
from an evaluation result as an attention field. ITiat is, since an evaluation means carries out evaluation 
which suited human being's subjectivity according to the physical feature, the attention field which 
suited himian being's subjectivity can be extracted. 

[0015] For example, when the physical feature includes whenever [ of a color / heterogeneous ], the 
degree of [ ] can be evaluated based on the difference in the color of each field. 
[0016] Moreover, since it includes [ formal ] whenever [ of a texture (pattern) / heterogeneous ] fiirther 
whenever [ of area ] heterogeneous [, and ] and whenever the color of the physical feature is 
heterogeneous in addition, if the degree of [ ** ] is evaluated based on whenever [ whenever / these / 
four / heterogeneous / at least one / heterogeneous ], according to the feature of a subject-copy image, 
the degree of [ ** ] can be evaluated exactly. 

[0017] Moreover, if it is the case where it evaluates also about three elements (a hue, saturation, 
lightness) of a color, the field near the conspicuous color (red) by himian being's subjectivity can be 
estimated as a most conspicuous field. 

[0018] Furthermore, if it evaluates also about the area of each field in spatial firequency or a subject- 
copy image, most conspicuous evaluation of a field can be judged still more exactly. 
[0019] Moreover, the image corresponding to the image which photoed the desired image with the 
photography means, for example, was photoed can be compounded, and a subject-copy image can also 
be generated. 

[0020] For example, if the location and height of a camera which are contained in a photography means 
are fixed, 360 degrees rotates and it enables it to take a photograph, when a cementation means joins the 
image in every frame, the subject-copy image of a panorama image is generable within the limits of 360 
degrees. 

[0021] With the automatic composition decision equipment of the 2nd invention, a maintenance means 
uses as a criteria image the pictures image and photograph corresponding to the photograph which the 
pictures which the painter drew, and a photographer took, and the data about this criteria image is held. 
Since a cutoff means cuts out the image of an attention field from a subject-copy image with reference to 
the data about this criteria image, it can determine the good composition of balance automatically. 
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[0022] Moreover, if the data about two or more criteria images is held, since a selection means can 
choose the composition suitable for the image of an attention field, the good composition of balance can 
be determined about all images. 

[0023] Since the data about the above criteria images contains the image data corresponding to a criteria 
image, the configuration data of a photographic subject, and the location data of a photographic subject 
at least, it chooses the composition suitable for an attention field, and can determine the good 
composition of balance automatically. That is, a photograph as if it photoed the photogrq)hic subject in 
the composition is generable. 
[0024] 

[Effect of the Invention] Since an attention field is extracted according to the evaluation result of the 
physical feature suitable for human being's subjectivity according to this invention, the attention field 
which suited human being's subjectivity can be extracted. 

[0025] According to other invention, since an attention field is cut out from a subject-copy image using 
the criteria composition of a criteria image, the good composition of balance can be determined 
automatically. 

[0026] The above-mentioned purpose of this invention, the other purposes, the feature, and an advantage 
will become still clearer from the detailed explanation of the following examples given with reference to 
a drawing. 
[0027] 

[Example] With reference to drawing 1 , the automatic composition decision equipment 10 of this 
example contains a video camera (only henceforth a "camera") 12. A camera station (a location and 
height) is fixed using a tripod etc., 360 degrees of cameras 12 rotate and they can be photoed. The image 
photoed with the camera 12 is inputted into image generation equipment 14, and the panorama image as 
a subject-copy iniage is generated from the photoed image. In addition, since 360 degrees of cameras 12 
can be rotated, image generation equipment 14 can generate a panorama image within the limits of 360 
degrees. 

[0028] Image generation equipment 14 generates a panorama image according to flow drawing shown in 
drawing 2 including video capture 16 and image cementation equipment 18. That is, if photography is 
started with a camera 12, after image generation equipment 14 starts processing, is step SI, carries out 
the capture of the first video frame (the 1st frame) using video capture 16 and generates a synthetic 
image from the image corresponding to oneth of them, it will generate a gray-scale image from the 
synthetic image using image cementation equipment 18. 

[0029] At continuing step S3, the capture of the following frame is carried out and a gray-scale image is 
generated like the 1st frame. At step S5, in order to determine from which location the synthetic image 
corresponding to the present frame (object frame) should be compounded, a retrieval template is started 
from the gray-scale image of an object frame as shown in drawing 3 . The width of face of this retrieval 
template is 50 pixels, and that height is the same as a frame image. Moreover, this retrieval plate is cut 
down from the zero of an object frame. 

[0030] Then, at step S7, a retrieval range is set up from the gray-scale image of a synthetic image, and 
the image of the same magnitude as a retrieval template is cut down from a retrieval range. That is, the 
width of face of a retrieval range is 1 00 pixels, and the height is the same as a frame image. Moreover, 
when the upper left of a frame image is made into a zero (x y) (= (1 1)), the x-coordinate of a retrieval 
plate is the location which subtracted only 100 fix)m the width of face of a frame image, and the y- 
coordinate is 1 . 

[0031] Next, in step S9, from a retrieval range, the image of the same magnitude as a retrieval template 
is cut down, and the absolute value of the difference of the mutual pixel value corresponding to the 
image and retrieval plate which were cut down is computed. At continuing step SI 1, it judges whether 
the difference of a pixel value is minimimi (0). If it is "YES" at step SI 1, the cut-down image will judge 
that it is the same magnitude as a retrieval plate, and will progress to step S13. On the other hand, if it is 
"NO" at step SI 1, after judging that the magnitude of the image and retrieval plate which were cut down 
differs and shifting the retrieval range of 1 pixel to the positive direction of a x axis at step SI 5, it will 
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return to step S5. Thus, processing is repeated until the image and retrieval template which were started 
become the same magnitude, shifting 1 pixel at a time. 

[0032] At step S 1 3, X coordinate in case the absolute value of a difference is minimum (0) is computed. 
Continuing step S 1 7 generates superposition and a new synthetic image for a synthetic image and an 
object frame image by the computed X coordinate. And at step SI 9, it judges whether the gray-scale 
image was generated from the generated synthetic image, and the panorama image was generated at step 
S21. If the panorama image will not be generated if it is "NO" at step S21 that is, it retxmis to step S3. 
On the other hand, if it is "YES" at step S21, it will judge that the panorama image (subject-copy image) 
as shown in drawingj4 (A) was generated, a subject-copy image will be outputted to the latter attention 
field extractor 20 ( drawing 1 ) at step S23, and processing will be ended. 

[0033] Retuming to drawing 1 , the attention field extractor 20 extracts a most conspicuous field 
(attention field) from the given subject-copy image. Specifically, the attention field extractor 20 
processes according to flow drawing shown in drawing 5 . That is, if a subject-copy image is given from 
image generation equipment 14, the attention field extractor 20 will start processing and will carry out 
field division of the subject-copy image at step S31. As shown in drawing 4 (B), specifically, a subject- 
copy image is divided into a drawing field and a picture field, the method of this field division — 
1997IEEE — setting — W.Y.Ma and B.S.Manjimath ** ~ the boimdary detection method based on "edge 
flow" indicated to "Edge Flow: A Framework of Boundary Detection and Image Segmentation" is 
applied. If it explains briefly, this method is edge flow which searches for the change direction of a color 
or a pattern in each location of an image, and consists of (size) in the strength of the change direction 
and change. A vector is determined. And it is edge flow by repetitive operation. A vector is spread in the 
direction of each vector, and the location where final vectors collide is made into the boundary line of 
each field. 

[0034] Therefore, the divided drawing field as shown in drawing 4 (C) is extracted, and step S33 
estimates the degree of a drawing field of [ ** ] at step S35. That is, it asks for the degree parameter of a 
drawing field of [ ** ]. Here, the subjectivity evaluation experiment which the artificer conducted 
showed that the physical features required for evaluation of the degree of [ ** ] were [ of a texture ] a 
color, spatial fi-equency, and area whenever [ formal ] heterogeneous, and whenever [ of area ] 
heterogeneous [, and ] whenever [ of a color ] heterogeneous. Moreover, generally, since the relation 
between human being's subjectivity evaluation result and a physical characteristic is expressed with a 
serpentine curve in many cases, an one ****** evening fimction is used for evaluation which is the 
degree of [ ** ] several 1. 
[0035] 
[Equation 1] 

[0036] The performance index of the degree of [ ** ] as shown in several 2 is defined using this beta 

fimction. 

[0037] 

[Equation 2] 
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[0038] Moreover, whenever [ above-mentioned of each field shown in several 2 / four ] heterogeneous, 
the degree HEX of [**] is defined by several 3. 

[0039] 
[Equation 3] 

RET = wh,'Hb, + wh.-HT,+wh,- HS,+wh,-HSh, 



HET, : fa«i<0*«1|»a* 

HC| 
HT. 
HS. 
HSh. 



«hi : S^fl^XK 

[0040] Furthermore, whenever [ of each physical feature ] heterogeneous, H is computed according to 
several 4, when standard deviation of dm and Difference d is set [ the difference of the feature value and 
the average feature value of all fields ] to std for the average of d and Difference d. 
[0041] 
[Equation 4] 

H(d.«ta..td) = |-^| 

[0042] That is, HC is CIE whenever [ of the color of each field ] heterogeneous. L* a* b"^ The result of 
having computed and computed the color difference of the average color of a field and the average color 
of all fields, a color difference average, and the standard deviation of the color difference using the color 
difference type in perception uniform color space is substituted for several 4, and it asks. In addition, the 
color difference type is indicated by 1994 "a color science handbook" at details. Specifically, it is the 
color information on each pixel of an image (R, G, B) Equal perceived-color-space L* a* b* It changes 
and Euclidean distance in ttie space is made into the color difference. That is, a color difference type is 
shown like several 5. 
[0043] 

[Equation 5] ^ 

AEab'=[(ALY+(Aa)%(Atf)r 

[0044] Next, HT is explained whenever [ of a texture ] heterogeneous, a texture — 1996IEEE — setting — 
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B.S.Manjunath W. Y.Ma ** ~ it is expressed by the textural facility vector indicated to "Texture 
Features for Browsing and Retrieval of Image Data", and the difference between textures is expressed 
by the Euclidean distance between textural facility vectors. 

[0045] Here, a textural facility vector is expressed by the vector which uses as an element the response 
when filtering an image by the Gabor filter bank constituted fi"om two or more Gabor filters with which 
magnitude differs firom a direction. However, since there is no orthogonality in each response when it 
filters with two or more Gabor filters, redundant information may be included in the filtered result. 
Then, the parameter of each filter in the Gabor filter bank is determined by the technique indicated by 
above-mentioned 1996IEEE. Specifically, it asks for theta, a, mhou, and mhov of the Gabor filter shovm 
by several 6 by several 7. In addition, for this technique, the filter which adjoins as shown in drawing 6 
is Half-Peak. In order to determine that the scale (magnitude) and bearing parameter of a filter touch and 
to express a textural facility, 24 filters (four scales, six directions) are used. 
[0046] 
[Equation 6] 

(L(x.y)=a^.G(x',y') 



y>=fedF:^0 -^)]-exp(27r jUx) 



U=a' 

x'=xcos0+ysine 
y'=-x8in0+ycoiBe 

i 



1 

^"^ 2Kcr. 

Gfnn(x. y) : X^-*ID=ni. ^ttlD=n«>^<Dii***-*7-(Jl.> 
m 
n 
S 
K 



0..S-1 
0..K-1 



[0047] 
[Equation 7] 
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a = (UL/U.)* 

, (a-1)U 
(a+l)VTIn2" 



<r.=U„(^)[u.-2.n.(sf2,n^-^»i]* 



ui: rr-w-rJi^A*»a>«ijxit>'&JBa* 

[0048] A textural facility vector is expressed by this several 6. Therefore, whenever [ of the texture of 
each field ] heterogeneous, HT is computed by substituting the average of distance with the mean vector 
of all fields, and distance, and the standard deviation of distance for several 4; 
[0049] Furthermore, whenever [ of the area of each field ] heterogeneous, HS is computed by 
substituting the average of a difference with the average area of all fields, and a difference, and the 
standard deviation of a difference for several 4. 

[0050] In whenever [ of the appearance of a field / heterogeneous ], and, whenever [ of the hole vsrhich a 
field includes / heterogeneous ], whenever [ of the form of each field ] heterogeneous, it unifies and HSh 
is obtained further again, as shown in several 8. 

[0051] 
[Equation 8] 
- SHP. +HODi 

tztiU 

HSh,: ««iO!)»fl>««* 
SHD.: WA'\(D9^»<OMA 
HOD,: mW(D^Ci^M 

[0052] It is necessary to be because for a graphic form to be rotated, to be visible, and to consider both 
differences of the direction as the difference in the configuration itself about the difference in a form, 
here. Then, a configuration can be described using the P type fourier descriptor which Yoshinori 
Kamisaka indicated to "the new fourier descriptor applicable also to an open curve" in 1984/3 "an 
institute-of-teleconimunications-engineers paper magazine." In this case, if the power to each firequency 
after the P type Fourier transform is detected, power is in agreement when the configuration of two 
graphic forms is the same. Moreover, if the fourier descriptor is used, it restricts to the time when a 
configuration and an angle of rotation are in agreement, and the fourier descriptor is in agreement. 
Therefore, the fourier descriptor and the power to each fi-equency can estimate how many configurations 
and appearance of two graphic forms are in agreement. 

[0053] As mentioned above, the featiu*e of an appearance can be expressed by the vector which consists 
of Fourier coefficients and power, and the difference of an appearance can be expressed using the 
Euclidean distance between these feature vectors. Therefore, whenever [ of the appearance of each 
field ] heterogeneous, HSh is computed by substituting the average of distance with the mean vector of 
all fields, and distance, and the standard deviation of distance for several 4. 

[0054] Next, Hho is explained whenever [ of a hole ] heterogeneous. In order to express the feature of 
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the hole which a field includes, it is necessary to take into consideration not only the configuration of a 
hole but the number and hole site of a hole. Asking for the primary moment of a field can define the 
difference in a hole site. In addition, the primary moment is indicated by 1992 "an image-analysis 
handbook." Therefore, the feature of a hole was expressed by the vector as shown in several 9. In 
addition, sequence of a hole is made into the order near the zero of an image. 
[0055] 
[Equation 9] 

no = [nh, M, ha 11 . • • • , hain . hb« , • • • , hbi • * • . ha .m . * • *, ha , hb,*wi » * • *. hb^w ] 

Mi: ««la>^>^>F 

hbuj: fflJ«ia>1«Ba);5a)j«a)n*7- 

[0056] Therefore, whenever [ of the hole of each field ] heterogeneous, Hho is computed by substituting 
the average of distance with the mean vector of all fields, and distance, and the standard deviation of 
distance for several 4. 

0057] Moreover, the degree FP of [ ****** ] shown in several 2 can be defined as shown in several 10. 
:0058] 

Equation 10] 

FPi.r=wf,-PG -i-wf,-PT+wf,-P&.t 

FP..t: fiiidi<DB$riflt(c^^(t«4«ftgiaS 
PC,: ||lAi(7)&a>ttSft 

PT, : faitti<o^mjaakft(oagit 
psu: iRm\(Dm9i(ommm 

[0059] In several 10, the degree PC of a color of [ ** ] is explained first. 

[0060] the conventional research ~ it is and 3 lightness in which the one where 2 saturation in which 1 
warm color is conspicuous from cold color is higher is conspicuous is reported for the higher one to be 
conspicuous. 1) is reported not to be dependent on a background color and for red to be a color with the 
high degree of [ ** ] by the experiment which **** and others conducted. Based on this result, in this 
example, it considers as the high hue whose red (0 R, G, B= 255, 0) is the degree of [ ** ] most, and it is 
assumed that the degree of [ ** ] is high, so that a hue is close to red. When above-mentioned 1992 "an 
image-analysis handbook" is asked for a hue here using the HSI congruence hexagon-head drill color 
model indicated by details, a red (0 R, G, B= 255, 0) hue is 0. That is, the point which a HSI congmence 
hexagon-head drill color model makes BK (black) a zero, and carries out a counter electrode to it is set 
to W (white). And the straight line which connects BK and W is a center mostly, and the hexagon which 
makes R (red), M (MAZENDA), B (blue), C (cyanogen), G (green), and Y (Hierro) top-most vertices so 
that it may intersect perpendicularly with the straight line is formed. Thus, the color space of the formed 
HSI congruence hexagon-head drill color model is used. In addition, the space of the rectangular three- 
dimensions system of coordinates used in order that a color space may display a perceived color as one 
point of space is said. That is, in this example, in order to search for a hue using a HSI congruence 
hexagon-head drill color model, the degree helium of a hue of [ ** ] becomes large, so that a hue is close 
to 0, For this reason, the degree helium of a hue of [ ** ] is computed by the formula shown in several 
11. 

[0061] 
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[Equation 11] 
He=1-i 

He: fiffiSISiS 

«a L. h> ?r <Dli«(*h=7r-hi: 6) 

[0062] Moreover, about 2, linearity-relation between saturation and the degree of [ ** ] is. Therefore, in 
this example, it was presupposed to the degree of saturation of [ ** ] that the saturation in a HSI 
congruence hexagon-head drill color model itself is used. 

[0063] Furthermore, about 3, Semmelroth showed that the relation shown by several 12 was realized in 
the research on the relation between human being's amount of sensation, and lightness. In addition, the 
relation shown in several 12 is shown by Daisen, hnai, Wake and others in 1996 "a new edition 
sensation / perception psychology handbook." 
[0064] 

[Equation 12] 

R=S"+klS-S.I" S^S. 
R = S"-klS-S.r s<s, 



R : 


(S%8 




S : 






S.: 






k : 






m : 






n : 







[0065] Moreover, in several 12, suiting well with human being's amount of sensation in the case of k= 
0.65, m= 0.4, and n= 0.2 was shown. Therefore, the formula shown by Semmelroth was made into the 
degree of lightness of [ ** ] in this example. 

[0066] Linear combination of the degree to three elements (a hue, saturation, lightness) of the above 
colors of [ ** ] is carried out using several 13, and the degree PC of a color of [ ** ] is defined. 
[0067] 

[Equation 131 
3 

fef£U 

He: MOtUBA 

s : i^Aomsm 

[0068] Next, the degree PT of a texture (spatial frequency) of [ ** ] is explained. 
[0069] In hirnian being's visual system, it is shown clearly that it has the property of a band pass 
machine in which sensitivity serves as max with specific spatial frequency. Until now, Kubota, 
Nishizawa and others are formulizing visual spatial frequency characteristics like several 14. In addition. 
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the formula of spatial frequency characteristics is indicated by Kubota and others in 1986/5 "an institute- 
of-telecommunications-engineers paper magazine" at "the three-dimension noise performance index of a 
television system, and application to the high-definition television." In addition, the unit of spatial 
frequency is changed into cpd (the number of cycles per 1 degree of vision) in several 14. 
[0070] 

[Equation 14] 
V = (X.Y)=A-n- 



V : «1CSJC 

Y : ^iSSMfflSAtt 

A: «l!£RJt®t--^t1lz««<br*fctoa)ff» 

[0071] The degree PT of a texture (spatial frequency) of [ ** ] is defined by the yision reaction V shown 
in this several 14. 

[0072] Then, the degree PS of area of [ ** ] is explained. 

[0073] It is said that the direction of a near object is conspicuous from the center of an image 
immediately after generally showing an image, and if a point of regard is moved, the direction of the 
object near the point of regard which moved is said for tihe degree of [ ** ] to become high. This is 
defined as the degree of a field of [ ** ]. Thus, whenever it keeps away from the point focusing on a 
certain point (point of regard), since it models that the degree of each point of [ ** ] beconles small 
gradually, the two-dimensional gauss function shown in several 15 is used. However, it is assumed that 
the center of a gauss function changes according to the time amount which the point of regard moved. 
[0074] 

[Equation 15] 
E(x,y. a,t)=-g^exp 

fc«U 

cx(t),cy(t): ^^mm 
a: /£*<y«» 

[0075] Here, it depends for the breadth coefficient mho of a gauss function on the distance d from a 
view to an image. That is, if distance d becomes large, the range which can be observed at once will 
become large. Generally, himian being's visible visual field is 20 - 30 degrees. Then, the breadth 
coefficient mho when setting the screen high of the image shown to H, and making a sight into d-H can 
be defined like several 16. In addition, Pix is the vertical number of pixels and theta is 20 - 30 degrees 
(0.176<tan theta / 2< 0.268). 
[0076] 

[Equation 16] 
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0'= 2-Pix-d-tan -5- 

[0077] Here, if it takes into consideration that it is the set whose field is a pixel, the degree PS of the 
area of a field of [ ** ] can be expressed by the sum of the degree of the field of the pixel of a field of 
[ ** ]. Therefore, the degree PS of area of [ * ] is defined like several 17. 
[0078] 

[Equation 17] 
fcffU 

RY. : mi<OBf«OYM«ia>IK^ 

[0079] Thus, it asks for the drawing field where step S37 estimates the degree of each drawing field of 
[ ** ], and the degree of [ ** ] serves as max at step S39 by the performance index of the defined degree 
of [ ** ]. That is, a most conspicuous field is determined as an attention field. Therefore, an attention 
field as shown in drawing 4 (D) can be extracted. 

[0080] In this example, in addition, each coefficient of the performance index of the degree of [ ** ] 
whl, wh2, wh3, [wh4] = [0,039, 0.010, 0,027, 0.020], and wfl, wf2 and = [0.132, 0.005, 0.100], and 
[wf3] [m, n] = [1 .358, 4.250] are used. Several 14 theta was made into 20 degrees, and Sight d was set to 
Im, and P type Fourier coefficients were carried out to to the 10th order. 

[0081] Moreover, although the degree of [ ** ] was evaluated using the eight above-mentioned physical 
features, this is for making two or more images which have all the features suit, and not necessarily 
needs to evaluate by this example about no physical features. 

[0082] Then, at step S41, it asks for the drawing field contiguous to the determined attention field, the 
color difference of an attention field and an adjoining-sheets field and the Euclidean distance of a 
textural facility vector are found, the thing whose color difference is less than 2.0 and whose EucUdean 
distance of a textural facility vector is less than 0.3 is extracted with an attention field, and processing is 
ended. In addition, the color difference is CIE as are mentioned above and shows several 5. L* a* b* It 
asks by the color difference type in perception uniform color space. Moreover, a textural facility vector 
is searched for according to several 6. 

[0083] It returns to drawing 1 and the attention field extracted with the attention field extractor 20 is 
given to composition cutoff equipment 22. Memory 24 is connected to composition cutoff equipment 
22, and two or more data related with the photograph which the picture which the painter drew, and the 
photographer took in memory 24 is memorized. Composition cutoff equipment 22 cuts out an attention 
field fi-om a subject-copy image with reference to the data memorized by memory 24 according to the 
composition of a painter or a photographer. Specifically, composition cutoff equipment 22 cuts off an 
attention field according to flow drawing shown in drawing 7 . That is, if an attention field and an 
adjoining drawing field are extracted by the attention field extractor 20, composition cutoff equipment 
22 starts processing, will be step S51 and will ask for the periphery pixel of an attention field. That is, it 
asks for the edge of the extracted attention field. At continuing step S53, by the technique of the paper 
which above-mentioned Kamisaka indicated, it asks for the P type Foxuier coefficients to the 10th order, 
and an attention field is made into a configuration vector. 

[0084] Here, the data memorized by memory 24 is the image data corresponding to criteria images, such 
as a pictures image and a photograph, the data of the configuration vector which extracted the 
photographic subject fi-om the pictures image and photograph, and described the periphery configuration 
of a photographic subject with P type Fourier coefficients, and location data corresponding to the 
positional information of a photographic subject as shown in drawing 8 . That is, the location data of a 
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photographic subject is data of the criteria composition of a criteria image. The location of the zero (top- 
most vertices nearest to the top-most vertices at the upper left of an image) of b and a circumscription 
rectangle for the length of the side of a and length (wl, hi), [ the length of the side beside the 
circumscription rectangle surrounding a photographic subject ] It is expressed with a vector like several 
1 8 when the location (location on the basis of the top-most vertices at the lower right of an image) of the 
end point (top-most vertices nearest to the top-most vertices at the lower right of an image) of a 
circumscription rectangle is set to (w2, h2). 
[0085] 

[Equation 18] Location data [kxl, kyl, kx2, ky2] = [wl /1 72 / h2 [ a and ]/b] [ a and hi ] [ b and w2 ] 
Then, the youg lid distance of the configuration vector searched for at step S53 by step S55 and the 
configuration vector of a pictures image or a photograph stored in memory 24 is found, that is, matching 
with the data of criteria composition is performed, and the data with which the configuration of a field 
was most similar at step S57 is acquired. In other words, the location data of the photographic subject of 
a pictures image with the smallest Euclidean distance is acquired. That is, the optimal image data for an 
attention field and an adjoining drawing field is chosen from two or more image data. And at step S59, it 
asks for the circvmiscription rectangle of an attention field, and some subject-copy images are cut out 
from the zero and end point of a circumscription rectangle according to several 19. In addition, by 
several 19, * means multiplication. 
[0086] 

[Equation 19] Width efface of Xl=xl-W*kxlYl=yl.H*kylX2=x2+W*kx2Y2^2+W*l^^ however 
the circxmiscription rectangle of an attention field is set to W, and (xl, yl ), and an end point are set 
[ height ] to (x2, y2) for the zero of H and a circumscription rectangle. Moreover, it considers as the 
height of a Y2= subject-copy image at the time of the width of face of X2= subject-copy image, and the 
height of a Y2> subject-copy image at the time of the width of face of Yl=l and X2> subject-copy 
image at the time of X 1=1 and YKl at the time of XKl. At this time, the top-most- vertices coordinate 
of the rectangle (image) to cut off is shown like several 20. 
[0087] 

[Equation 20] top-most-vertices coordinate =[(X1, Yl) of the image to cut off, (XI, Y2), and (X2, Yl) - 
-](X2,Y2) 

Then, the result (image) cut off at step S61 is outputted, and processing is ended. 

[0088] Therefore, according to the composition of a pictures image as shown in drawing 9 (A), an 

attention field and an adjoining drawing field as shown in drawing 9 (B) can be cut off. 

[0089] Since the field (attention field) which is most conspicuous fi-om a subject-copy image according 

to the physical feature which was mentioned above is extracted according to this example, the attention 

field which suited himian being's subjectivity can be extracted. 

[0090] Moreover, since the extracted attention field is cut off according to the composition of the 
photograph which the pictures which the painter drew, and a photographer took, a photograph as if it 
photoed the photographic subject in the composition is generable. That is, the good composition of 
balance can be determined automatically. 

[0091] In addition, since this attention field extractor suits human being's subjectivity and can ask for the 
degree of an image field of [ ** ], it is applicable to equipment which performs objective evaluation of 
each field of the image used as the candidate for evaluation, or an image which is conspicuous and 
performs weighting to that field according to whenever (the degree of [ ** J), for example in the quality 
evaluation of a digital compression image and an image. 

[0092] Moreover, in the printing field, it is applicable to the equipment which can judge automatically 
the field which can make light of problems, such as a printing gap, and the field which is not so 
according to the degree of [ ** ] in automation of quality control. [0093] Furthermore, in the poster work 
used for the design field, especially an advertisement, it is applicable to the equipment by which it 
evaluates objective whether it is that the portion against which a company wants to appeal most is 
conspicuous. 

[0094] Moreover, if it controls rotation of a camera, a tilt, and a zoom using an attention field extractor 
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in being the dress with which a user is conspicuous, a camera can run after a user automatically. A user's 
snapshot can be taken, if it follows, for example, a shutter is tumed off by predetermined timing. 
Furthermore, the snapshot of the good user of balance can be created by determining composition 
automatically using composition decision equipment from the photoed image (subject-copy image). 

[Translation done.] 
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♦NOTICES* 

Japan Patent: Office is not responsible for any 
damages caused, by the use of this translation. 

l.This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2^4c3ic4c shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 11 It is illustration drawing showing one example of this invention. 

[Drawing 2] It is flow drawing showing a part of processing of the image generation equipment shown 
in the drawing 1 example. 

[Drawing 31 It is illustration drawing showing how to compoimd a subject-copy image with the image 
cementation equipment of the drawing_l example. 

[Drawing 4] It is flow drawing showing a part of processing of the attention field extractor shown in the 
drawing 1 example. 

[Drawing 5] It is illustration drawing showing how to extract an attention field according to flow 
drawing shown in drawing 4 . 

[Drawing 6] It is illustration drawing showing the Gabor filter. 

[Drawing 7] It is flow drawing showing a part of processing of the composition cutoff equipment shown 
in the drawing 1 example. 

[Drawing 8] It is illustration drawing showing the positional information in the case of cutting off an 
image according to flow drawing shown in drawing 7 . 

[Drawing 9] It is illustration drawing showing pictures image and image referred to and cut off of the 
painter who refers to when cutting off an image according to flow drawing shown in drawing 7 , 
[Description of Notations] 

10 — Automatic Composition Decision Equipment Using Attention Field Extractor 
12 — Camera 

14 - Image Generation Equipment 

16 -Video Capture 

1 8 - Image Cementation Equipment 

20 - It is Extractor whenever [ Attention ]. 

22 - Composition Cutoff Equipment 

24 — Memory 



[Translation done.] 
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♦NOTICES* 

Japan Patent: Office is not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2. **** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim 1] An attention field extractor equipped with an evaluation means to be the attention field 
extractor which extracts an attention field fi-om a subject-copy image, and to evaluate the degree of 
[ ** ] based on the physical feature, and an extract means to extract said attention field according to an 
evaluation result of said evaluation means. 

[Claim 2] Said physical feature is an attention field extractor including whenever [ of a color / 
heterogeneous ] according to claim 1. 

[Claim 3] For said evaluation means, said physical feature is an attention field extractor according to 
claim 2 by which said degree of [ ** ] is evaluated based on whenever [ at least one / heterogeneous ] in 
whenever [ four ] heterogeneous, including whenever [ of a texture whenever / of area / heterogeneous / 
heterogeneous ] fiuther, whenever [ formal ] heterogeneous. 

[Claim 4] Said physical feature is an attention field extractor according to claim 1 to 3 which contains a 
color fiirther. 

[Claim 5] Said physical feature is an attention field extractor according to claim 4 which contains fiuther 
area and spatial fi-equency of a field in said subject-copy image. 

[Claim 6] An attention field extractor according to claim 1 to 5 fiuther equipped with a photography 
means to photo a desired image, and an image generation means to generate said subject-copy image 
based on said image. 

[Claim 7] Said image generation means is an attention field extractor including a cementation means to 
join said image photoed with said camera for every fi^ame according to claim 6, including a driving 
means to which said photography means carries out the rotation drive of a camera and said camera. 
[Claim 8] Automatic composition decision equipment equipped with a maintenance means to hold data 
about a criteria image which is automatic composition decision equipment using an attention field 
extractor according to claim 1 to 7, and has criteria composition, and a cutoff means which cuts out an 
image of an attention field fi"om a subject-copy image with reference to said criteria composition. 
[Claim 9] Said maintenance means is automatic composition decision equipment according to claim 8 
which holds data about said two or more criteria images, and is fiuther equipped with a selection means 
to choose said data which siiited an image of said attention field. 

[Claim 10] Said data is automatic composition decision equipment according to claim 8 or 9 which 
contains image data corresponding to said criteria image, configuration data of a photographic subject, 
and location data of a photographic subject at least. 



[Translation done.] 
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[Drawing 4] 
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[Drawing 2] 
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[Drawing 51 
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[Drawing 7] 
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